A Multi objective Approach to Evolving Artificial Neural Networks for Coronary Heart Disease Classification

The optimisation of the accuracy of classifiers in pattern recognition is a complex problem that is often poorly understood. Whilst numerous techniques exist for the optimisa- tion of weights in artificial neural networks (e.g. the Widrow-Hoff least mean squares algorithm and back propagation techniques), there do not exist any hard and fast rules for choosing the structure of an artificial neural network - in particular for choosing both the number of the hidden layers used in the network and the size (in terms of number of neurons) of those hidden layers. However, this internal structure is one of the key factors in determining the accuracy of the classification. This paper proposes taking a multi-objective approach to the evolutionary design of artificial neural networks using a powerful optimiser based around the state-of-the-art MOEA/D- DRA algorithm and a novel method of incorporating decision maker preferences. In contrast to previous approaches, the novel approach outlined in this paper allows the intuitive consideration of trade-offs between classification objectives that are frequently present in complex classification problems but are often ignored. The effectiveness of the proposed multi-objective approach to evolving artificial neural networks is then shown on a real-world medical classification problem frequently used to benchmark classification method

Multi-objective evolution of artificial neural networks in multi-class medical diagnosis problems with class imbalance

This paper proposes a novel multi-objective optimisation approach to solving both the problem of finding good structural and parametric choices in an ANN and the problem of training a classifier with a heavily skewed data set. The state-of-the-art CMA-PAES-HAGA multi-objective evolutionary algorithm [41] is used to simultaneously optimise the structure, weights, and biases of a population of ANNs with respect to not only the overall classification accuracy, but the classification accuracies of each individual target class. The effectiveness of this approach is then demonstrated on a real-world multi-class problem in medical diagnosis (classification of fetal cardiotocograms) where more than 75% of the data belongs to the majority class and the rest to two other minority classes. The optimised ANN is shown to significantly outperform a standard feed-forward ANN with respect to minority class recognition at the cost of slightly worse performance in terms of overall classification accuracy

CMA-PAES: Pareto archived evolution strategy using covariance matrix adaptation for multi-objective optimisation

The quality of Evolutionary Multi-Objective Optimisation (EMO) approximation sets can be measured by their proximity, diversity and pertinence. In this paper we introduce a modular and extensible Multi-Objective Evolutionary Algorithm (MOEA) capable of converging to the Pareto-optimal front in a minimal number of function evaluations and producing a diverse approximation set. This algorithm, called the Covariance Matrix Adaptation Pareto Archived Evolution Strategy (CMA-PAES), is a form of (μ + λ) Evolution Strategy which uses an online archive of previously found Pareto-optimal solutions (maintained by a bounded Pareto-archiving scheme) as well as a population of solutions which are subjected to variation using Covariance Matrix Adaptation. The performance of CMA-PAES is compared to NSGA-II (currently considered the benchmark MOEA in the literature) on the ZDT test suite of bi-objective optimisation problems and the significance of the results are analysed using randomisation testing. © 2012 IEEE

Preference focussed many-objective evolutionary computation

Solving complex real-world problems often involves the simultaneous optimisation of multiple con icting performance criteria, these real-world problems occur in the elds of engineering, economics, chemistry, manufacturing, physics and many more. The optimisation process usually involves some design challenges in the form of the optimisation of a number of objectives and constraints. There exist many traditional optimisation methods (calculus based, random search, enumerative, etc...), however, these only o er a single solution in either adequate performance in a narrow problem domain or inadequate performance across a broad problem domain. Evolutionary Multi-objective Optimisation (EMO) algorithms are robust optimisers which are suitable for solving complex real-world multi-objective optimisation problems, as they are able to address each of the con icting objectives simultaneously. Typically, these EMO algorithms are run non-interactively with a Decision Maker (DM) setting the initial parameters of the algorithm and then analysing the results at the end of the optimisation process. When EMO is applied to real-world optimisation problems there is often a DM who is only interested in a portion of the Pareto-optimal front, however, incorporation of DM preferences is often neglected in the EMO literature. In this thesis, the incorporation of DM preferences into EMO search methods has been explored. This has been achieved through the review of EMO literature to identify a powerful method of variation, Covariance Matrix Adaptation (CMA), and its computationally infeasible EMO implementation, MO-CMA-ES. A CMA driven EMO algorithm, CMA-PAES, capable of optimisation in the presence of many objectives has been developed, benchmarked, and statistically veri ed to outperform MO-CMA-ES and MOEA/D-DRA on selected test suites. CMA-PAES and MOEA/D-DRA with the incorporation of the novel Weighted Z-score (WZ) preference articulation operator (supporting a priori, a posteriori or progressive incorporation) are then benchmarked on a range of synthetic and real-world problems. WZ-CMA-PAES is then successfully applied to a real-world problem regarding the optimisation of a classi er for concealed weapon detection, outperforming previously published classi er implementations

Age Detection Through Keystroke Dynamics From User Authentication Failures

In this paper an incident response approach is proposed for handling detections of authentication failures in systems that employ dynamic biometric authentication and more specifically keystroke user recognition. The main component of the approach is a multi layer perceptron focusing on the age classification of a user. Empirical findings show that the classifier can detect the age of the subject with a probability that is far from the uniform random distribution, making the proposed method suitable for providing supporting yet circumstantial evidence during e-discovery

Evaluation of Mental Workload and Familiarity in Human Computer Interaction with Integrated Development Environments using Single-Channel EEG

With modern developments in sensing technology it has become possible to detect and classify brain activity into distinct states such as attention and relaxation using commercially avail- able EEG devices. These devices provide a low-cost and minimally intrusive method to observe a subject’s cognitive load whilst interacting with a computer system, thus providing a basis for deter- mining the overall effectiveness of the design of a computer interface. In this paper, a single-channel dry sensor EEG headset is used to record the mental effort and familiarity data of participants whilst they repeat a task eight times in either the Visual Studio or Eclipse Integrated Development Environments (IDEs). This data is used in conjunction with observed behaviour and perceived difficulties reported by the participants to suggest that human computer interaction with IDEs can be evaluated using mental effort and familiarity data retrieved by an affordable EEG headse

Generalised Pattern Search Based on Covariance Matrix Diagonalisation

Pattern Search is a family of gradient-free direct search methods for numerical optimisation problems. The characterising feature of pattern search methods is the use of multiple directions spanning the problem domain to sample new candidate solutions. These directions compose a matrix of potential search moves, that is the pattern. Although some fundamental studies theoretically indicate that various directions can be used, the selection of the search directions remains an unaddressed problem. The present article proposes a procedure for selecting the directions that guarantee high convergence/high performance of pattern search. The proposed procedure consists of a fitness landscape analysis to characterise the geometry of the problem by sampling points and selecting those that whose objective function values are below a threshold. The eigenvectors of the covariance matrix of this distribution are then used as search directions for the pattern search. Numerical results show that the proposed method systematically out-performs its standard counterpart and is competitive with modern complex direct search and metaheuristic methods

Assessing performance of artificial neural networks and re-sampling techniques for healthcare datasets.

Re-sampling methods to solve class imbalance problems have shown to improve classification accuracy by mitigating the bias introduced by differences in class size. However, it is possible that a model which uses a specific re-sampling technique prior to Artificial neural networks (ANN) training may not be suitable for aid in classifying varied datasets from the healthcare industry. Five healthcare-related datasets were used across three re-sampling conditions: under-sampling, over-sampling and combi-sampling. Within each condition, different algorithmic approaches were applied to the dataset and the results were statistically analysed for a significant difference in ANN performance. The combi-sampling condition showed that four out of the five datasets did not show significant consistency for the optimal re-sampling technique between the f1-score and Area Under the Receiver Operating Characteristic Curve performance evaluation methods. Contrarily, the over-sampling and under-sampling condition showed all five datasets put forward the same optimal algorithmic approach across performance evaluation methods. Furthermore, the optimal combi-sampling technique (under-, over-sampling and convergence point), were found to be consistent across evaluation measures in only two of the five datasets. This study exemplifies how discrete ANN performances on datasets from the same industry can occur in two ways: how the same re-sampling technique can generate varying ANN performance on different datasets, and how different re-sampling techniques can generate varying ANN performance on the same dataset

Review of underground hydrogen storage: Concepts and challenges

The energy transition is the pathway to transform the global economy away from its current dependence on fossil fuels towards net zero carbon emissions. This requires the rapid and large-scale deployment of renewable energy. However, most renewables, such as wind and solar, are intermittent and hence generation and demand do not necessarily match. One way to overcome this problem is to use excess renewable power to generate hydrogen by electrolysis, which is used as an energy store, and then consumed in fuel cells, or burnt in generators and boilers on demand, much as is presently done with natural gas, but with zero emissions. Using hydrogen in this way necessitates large-scale storage: the most practical manner to do this is deep underground in salt caverns, or porous rock, as currently implemented for natural gas and carbon dioxide. This paper reviews the concepts, and challenges of underground hydrogen storage. As well as summarizing the state-of-the-art, with reference to current and proposed storage projects, suggestions are made for future work and gaps in our current understanding are highlighted. The role of hydrogen in the energy transition and storage methods are described in detail. Hydrogen flow and its fate in the subsurface are reviewed, emphasizing the unique challenges compared to other types of gas storage. In addition, site selection criteria are considered in the light of current field experience.Cited as: Hematpur, H., Abdollahi, R., Rostami, S., Haghighi, M., Blunt, M. J. Review of underground hydrogen storage: Concepts and challenges. Advances in Geo-Energy Research, 2023, 7(2): 111-131. https://doi.org/10.46690/ager.2023.02.0